Robot controlled by wireless signals

ABSTRACT

A user communicates with a companion with his or her portable telephone attached to a telephone attachment pocket of a robot. The robot acts in accordance with a voice signal which is outputted from the portable telephone, thereby behaving as if it were speaking to the user. The robot outputs a voice in response to the user&#39;s voice which is picked up by a microphone, thereby behaving as if it were listening to the user. Owing to this robot or a robot system using the same, the user feels less odd in his or her communication using the portable telephone.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a robot for supporting user's dailylife and a robot system which uses such a robot.

2. Description of the Related Art

A robot has been proposed which presents a shape and actions resemblingthose of pet animals or imaginary beings and which makes a user feelclose to the robot. Such a robot with a dedicated operation programinstalled is capable of moving movable parts or outputting a voice inaccordance with information such as a voice picked up through amicrophone and images acquired with a CCD camera.

A communication apparatus such as a telephone is used for the purpose oftalking with someone who is far away. Such a communication apparatus isstructured so as to output the companion's voice. A dialogue with thecompanion using a communication apparatus is conducted generally withoutuser's visual confirmation of the companion. In the recent years, arobot has been proposed which analyzes a speaker's voice and behaves asif it were a listener. Acting in response to the speaker's (user's)voice, such a robot makes the user feel as if the user is inconversational communication with the robot.

However, on the occasion of conversation between human beings usingcommunication apparatuses such as telephones, the conventional robotdescribed above can not act in response to a voice from the person onthe phone (speaker). Further, the user (listener) can not visuallyconfirm the companion. Hence, the robot above makes the user feel odd.

Further, the conventional robot can not notify a user who is far away ofthe state inside the house. Hence, to allow the user to monitor insidethe house, it is necessary to install an expensive monitoring system.

In addition, although the operation program installed in theconventional robot can be modified on a minor scale, the conventionalrobot can not operate based on a totally new operation program.

Further, as an operation program to be realized is largely modified inthe conventional robot, a controller controlling intended actions cannot remain accommodating, and therefore, desired actions can not berealized unless almost all structural elements are modified.

Further, the conventional robot is not capable of outputting a voice orbehaving in accordance with the user's mental condition, such as happy,sad or angry feeling.

Apparatuses installed in a house, such as an air conditioner, atelevision set and a washing machine, are equipped with indicating lampsfor notifying the residents of malfunction. However, to notify theresidents of malfunction, the residents need come close to theseapparatuses.

Further, in a house, owing to the resident's habit in his or hereveryday life, the apparatuses which are set within the house aremanipulated in most cases in an approximately regular mannerapproximately at the regular time, e.g., an air conditioner is activatedat about 9 p.m. every day. To operate the apparatuses, the residentneeds manipulate the apparatuses every time.

BRIEF SUMMARY OF THE INVENTION

The present invention has been made with the aim of solving the aboveproblem, and an object thereof is to provide a robot which behaves as ifit were a companion in conversation while looking at a user (listener)and therefore does not make the user feel odd when communicating withthe user, and to provide a robot system using such a robot.

Another object of the present invention is to provide a robot which actsin response to a voice of a person on the phone and accordingly behavesas if it were a speaker when a voice or a voice signal is outputted froma communication apparatus, while appropriately responds to what a usersays and accordingly behaves as if it were a listener when the user'svoice is received by a voice input unit, and to provide a robot systemusing such a robot.

Still another object of the present invention is to provide a robotwhich is capable of acting in accordance with instructions from anexternal communication apparatus and notifying a user who is far away ofthe state within a house, and to provide a robot system using such arobot.

Further object of the present invention is to provide a robot which iscapable of downloading a CM-operation program registered in an externalserver apparatus through a communication network, e.g., on Internetlines.

Yet further object of the present invention is to provide a robot whichis capable of easily realizing a hardware structure designed for adesired application by modifying an FPGA program by means of ageneral-purpose hardware structure, in accordance with a control mode,such as torque control and speed control, of a motor required for theapplication.

Another object of the present invention is to provide a robot whichoutputs a voice which matches with a mental condition of a user inuser's life and accordingly makes the user feel close to the robot, andto provide a robot system using such a robot.

Yet another object of the present invention is to provide a robot whichmanipulates external apparatuses in accordance with the user's habit inhis or her everyday life so that the robot saves the trouble for theuser even if the user does not manipulate the external apparatuses in anapproximately regular manner approximately at the regular time everyday.

A robot system according to the present invention comprises: a robot;and a communication apparatus which receives a voice signal and outputsa voice or voice signal which corresponds to the received voice signal.The robot comprises: at least one movable unit; an imaging unit; and aninput unit which accepts the voice or voice signal from thecommunication apparatus which receives the voice signal and outputs thevoice or voice signal which corresponds to the received voice signal.The robot comprises a control unit which drives the movable unit basedon the voice or voice signal inputted at the input unit and outputtedfrom the communication apparatus and an image captured by the imagingunit.

In this robot or robot system, the control unit drives the movable unitof the robot in a controlled manner based on the voice or voice signaloutputted from the communication apparatus such as a portable telephone,and the image captured by the imaging unit. Hence, the robot can behaveas if it were a companion to a user (listener) while looking at theuser, which makes the user feel less odd during the communication.

A robot according to the present invention comprises: at least onemovable unit; a voice input unit; a voice output unit; and an input unitwhich accepts a voice or voice signal from a communication apparatuswhich receives a voice signal and outputs a voice or voice signal whichcorresponds to the received voice signal. The robot comprises: a controlunit which drives the movable unit based on the voice or voice signaloutputted from the communication apparatus; and a voice determining unitwhich determines a voice to be outputted based on the voice which isinputted at the voice input unit. The voice output unit outputs thevoice which is determined by the voice determining unit.

In this robot or robot system, the control unit drives the movable unitof the robot in a controlled manner based on the voice or voice signaloutputted from the communication apparatus. Hence, the robot can behaveas if it were a speaker. Further, in this robot or robot system, thevoice determining unit determines a voice which corresponds to the voicewhich is inputted at the voice input unit, and the voice output unitoutputs thus determined voice. Hence, the robot can behave as if it werea listener, responding appropriately to the user who is speaking to therobot. As a result, not only when the user is speaking to the user butalso when the user is communicating with someone else, it is possible tomake the user feel as if the user were having conversation with therobot and hence make the user feel less odd.

A robot system according to the present invention comprises: the robot;and the communication apparatus which receives the voice signal andoutputs the voice or voice signal which corresponds to the receivedvoice signal. The robot further comprises an imaging unit, and thecontrol unit drives the movable unit based on an image captured by theimaging unit.

In this robot and the robot system using this robot, the imaging unitcaptures an image around the robot, and the movable unit is driven in acontrolled manner based on thus obtained image such that the user islocated at the center in the image. Hence, it is possible to have therobot faced to the user, and therefore, both when the user is speakingto the robot and when the user is communicating with someone else, it ispossible to make the user feel much less odd.

A robot system according to the present invention comprises the robotand the communication apparatus, and the communication apparatuscomprises: a first receiving unit which receives from outside firstaction instructing information which expresses an instruction regardingan action of the movable unit; a first transmission unit which transmitsto the robot second action instructing information which corresponds tothe first action instructing information which is received by the firstreceiving unit; a second receiving unit which receives first imageinformation which expresses an image which is transmitted from therobot; and a second transmission unit which transmits to outside secondimage information which corresponds to the first image information whichis received by the second receiving unit. The robot comprises: at leastone movable unit; an imaging unit; a third receiving unit which receivesthe second action instructing information from the communicationapparatus; a control unit which drives the movable unit based on thesecond action instructing information which is received by the thirdreceiving unit; and a third transmission unit which transmits the firstimage information to the communication apparatus based on the imagecaptured by the imaging unit.

In the robot and the robot system, the first action instructinginformation which instructs an action of the robot is transmitted fromoutside, the first action instructing information is received by thecommunication apparatus which is installed within a building andtransmitted to the robot as the second action instructing information(action instructing information), and each movable unit of the robot isdriven in a controlled manner based on the second action instructinginformation (action instructing information). Hence, a user who is faraway can remotely operate the robot from outside. Further, in the robotand the robot system, the image captured by the imaging unit of therobot is transmitted as the first image information (image information)from the robot to the communication apparatus, and the second imageinformation corresponding to the first image information (imageinformation) is transmitted to outside from the communication apparatus.Hence, a user who is far away can get notified of the state inside hisor her house. This allows the user to monitor inside the house withoutintroducing an expensive monitoring system.

The robot according to the present invention acquires an operationprogram to a memory from an external server apparatus through acommunication network.

A desired CM-operation program can be easily loaded into the robot froma web page of a company for instance. In addition, the robot can expressinformation based on the CM-operation program as an action thereof.

The robot according to the present invention comprises a programmablecontroller which can modify by means of program a hardware structurewhich is for supplying a control signal to a drive circuit.

By modifying a program of the programmable controller, the drive circuitof a motor which operates the movable units of the robot can easily setup control of voltage, control of current, etc. Hence, it is easy tomodify torque control, speed control and the like of the motor inaccordance with a purpose.

A robot according to the present invention comprises: a receiving unitwhich receives information from outside; a voice determining unit whichdetermines a voice based on the information received by the receivingunit; and a voice output unit which outputs the voice determined by thevoice determining unit. The information contains information regardingeveryday life of a target person.

In this robot, information corresponding to the condition of the user(mental condition) collected by a portable apparatus which the userbrings with him or her is received by the receiving unit, and the voicedetermining unit determines a voice based on this information, andtherefore, it is possible to output a voice which fits with the user'smental condition. Hence, when the target's (user's) heart rate is highfor instance, it is possible to determine that the user is feelingtense. When the user's portable telephone has received many incomingcalls, it is possible to determine that the user is feeling busy.Further, the robot determines a voice based on information regardingeveryday life of a target person (user). Hence, when the user has walkedmany steps in one day for example, the robot outputs a voice praisingthe user and makes the user feel pleased and even closer to the robot.

A robot according to the present invention comprises: a memory unitwhich stores manipulation history information regarding the history ofmanipulation given by a target person to an external apparatus; and anoperation unit which manipulates the external apparatus based on themanipulation history information stored in the memory unit.

The robot can operate the external apparatus in accordance with thetarget's (user's) habits in his or her everyday life. The user does notneed to manipulate the external apparatus through approximately constantmanipulation approximately at the same time of a day every day, andthus, and the robot saves the troubles for the user.

A robot according to the present invention further comprises: areceiving unit which receives the information transmitted from theexternal apparatus; and a voice output unit which outputs a voice basedon the information received by the receiving unit. The operation unitmanipulates the external apparatus when the manipulation historyinformation is information which indicates that information was receivedfrom a particular sender approximately at the same time of a day over acouple of days, so that the external apparatus receives information fromthe particular sender at the time corresponding to the time of a dayafter these past few days and the external apparatus transmits thusreceived information .

Even when the user does not manipulate the external apparatus, the robotcollects information from the particular sender at the time of a dayevery day in accordance with the user's habits in his or her everydaylife and the robot notifies the user of this by means of a voice. Hence,it is not necessary for a user to manipulate the external apparatus toreceive information from the particular sender at the time of a dayevery day, whereby the trouble is saved for the user.

The above and further objects and features of the invention will morefully be apparent from the following detailed description withaccompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a perspective view showing an essential structure of a firstembodiment of a robot according to the present invention;

FIG. 2 is a block diagram showing the essential structure of the firstembodiment of the robot according to the present invention;

FIG. 3 is a block diagram showing a hardware structure of a controlapparatus of a robot which is capable of modifying an operation program;

FIG. 4 is an explanatory diagram of functions of a CPU;

FIGS. 5 and 6 are flow charts showing the sequence of processing in theCPU for setting up of an operation mode of the robot according to thefirst embodiment;

FIG. 7 is a flow chart showing the sequence of processing in the CPU forcontrol of actions by the robot according to the first embodiment;

FIG. 8 is a flow chart showing the sequence of processing in a nodtiming detector 121;

FIG. 9 is a flow chart showing the sequence of processing in an emotionmodel generator 140;

FIG. 10 is a reply operation table for describing operations of a replyoperation controller 160;

FIG. 11 is a schematic diagram showing an essential structure of anautomatic distribution/receipt system for a robot operation programaccording to the present invention;

FIG. 12 is a flow chart showing the flow of automatic download of anoperation program by a communication terminal and a robot;

FIG. 13 is a flow chart showing the flow of operations by an externalserver apparatus;

FIG. 14 is an operation flow chart showing a hardware configuration of acontrol apparatus of a robot which is capable of modifying an operationprogram;

FIG. 15 is a schematic diagram showing an essential structure of asecond embodiment of the robot according to the present invention;

FIG. 16 is a block diagram showing a structure of the robot according tothe second embodiment;

FIG. 17 is a chart showing one example of command data for a robot;

FIG. 18 is a block diagram showing a structure of a computer;

FIGS. 19, 20, 21 and 22 are flow charts showing the flow of actions in arobot system according to the second embodiment;

FIG. 23 is a perspective view showing an essential structure of a thirdembodiment of the robot according to the present invention;

FIG. 24 is a block diagram showing the essential structure of the thirdembodiment of the robot according to the present invention;

FIG. 25 is a flow chart showing the sequence of processing in a CPU ofthe third embodiment of the robot according to the present invention;

FIG. 26 is a flow chart showing the sequence of first control processingby a CPU;

FIG. 27 is a flow chart showing the sequence of second controlprocessing by the CPU;

FIG. 28 is a flow chart showing the sequence of third control processingby the CPU;

FIG. 29 is a block diagram showing an essential structure of a fourthembodiment of the robot according to the present invention; and

FIG. 30 is a flow chart showing one example of the sequence ofprocessing in a CPU in which a robot remotely manipulates a televisionset.

DETAILED DESCRIPTION OF THE INVENTION

The following description will explain the present invention in detailwith reference to the drawings illustrating some embodiments thereof.

FIRST EMBODIMENT

FIGS. 1 and 2 are a perspective view and a block diagram showing anessential structure of a first embodiment of a robot according to thepresent invention. In FIGS. 1 and 2, denoted at 1 is a robot and denotedat 2 is a portable telephone machine (communication apparatus). Therobot 1 has an outer shell of plastic or fabric, etc., and a shaperesembling a pet animal, an imaginary animal or the like. The robot 1 isformed by a head portion 11 and a trunk portion 12 and internallycomprises operation mechanisms (not shown) which include a plurality ofmotors M, M, . . . M for a nodding action in which the head portion 11sways in an up/down direction, a shaking action in which the headportion 11 shakes in a left/right direction, an open/close action inwhich a mouth 11 a formed at the head portion 11 opens and closes, anaction of eye balls 11 b and 11 b formed at the head portion 11 in whichthe eye balls 11 b and 11 b roll in the up/down direction and theleft/right direction, a blinking action in which eyelids 11 c and 11 cswing in the up/down direction and a traveling action in which the robot1 travels as wheels not shown disposed to the trunk portion 12 roll.

A hardware structure of a control apparatus of the robot which iscapable of modifying an operation program will be described withreference to FIG. 3. In FIG. 3, the control apparatus comprises a maincontrol board 25 having a capability of high-order calculation and aplurality of outer connection boards 32, 32.

The main control board 25 seats a CPU 16, a communication interface 20which enables connection with external equipment, a network or apersonal communication terminal 27 such as a portable telephone, aprogrammable device 29 which is bus-connected with the plurality ofouter connection boards 32 and whose internal logic calculation circuitcan be re-written by the CPU 16 by means of the configuration, and amemory 31 which is connected with the CPU 16.

The programmable device 29 is an FPGA (Field Programmable Gate Array) orthe like whose internal architecture is an SRAM structure for instance,and is commercially available from ALTERA, XILINX or the like.

Meanwhile, the memory 31 is formed by a non-volatile memory, such as aflash memory, which stores data needed to configure the FPGA 29, theprogram in the CPU 16 and the like, and a RAM which stores data whichare used by the CPU 16 for execution of the program.

Each outer connection boards 32 bus-connected with the main controlboard 25 mounts an FPGA 26 which serves as a programmable device, andthe internal logic calculation circuit of the FPGA 26 is formed inaccordance with configuration data transmitted through a dedicatedcontrol signal bus 28.

A connection bus 30 which connects the FPGA 29 and 26 mounted on themain control board 25 and the respective outer connection boards 32 toeach other includes data, a control signal and a power source line.

The outer connection boards 32 mounts a general-purpose sensor interface36 which is capable of accepting digital pulses and analog voltagesignals from a potentiometer and other force sensors which measure adeviation at the last stage through an encoder 33, which is connectedwith the motors M which activate operational portions of the robot forexample, a reducer and the like. It is via this sensor interface 36 thatcontrol signals are accepted by the FPGAs 26.

Connected between each motor M and each FPGA 26 is a drive circuit 17which controls a voltage to be applied to the motor M based on aninstruction value from the FPGA 26. The drive circuit 17 uses PWMcontrol or the like which requires repeated switching at a predeterminedfrequency (about 16 kHz) from the FPGA 26 and allows to control a dutywithin one cycle (a ratio of the ON-state in which a voltage is applied,to the OFF-state in which a voltage is not applied).

Having the bus structure denoted at the dotted and the solid lines inFIG. 3, a plurality of outer connection boards 32 can be connected whichhave the same structure in accordance with the number of the motors Mused by the application. In this case, IDs for distinguishing the boardscan be set by disposing a jumper pin on the board or directly writingthe IDs in the configuration data of the FPGAs 26.

Further, the outer connection boards 32 can connect not only with boardsfor driving the motors and having the same structure but with imageinput and output boards, voice input and output boards and the like inaccordance with the necessity associated due to the application.

A telephone attachment pocket 12 a which is to receive a portabletelephone 2 is formed in a side portion of the trunk portion 12. In thevicinity of the telephone attachment pocket 12 a, there is a connectingterminal 13 which is connected with a connecting terminal 21 disposed toa side portion of the portable telephone 2. The connecting terminal 13is connected with the CPU 16 via an amplifier 14 a and an AID converter15 a. The amplifier 14 a is connected with another amplifier 14 c aswell, and the amplifier 14 c is connected with a speaker 12 c which isdisposed to the side portion of the trunk portion 12.

Owing to this, a voice signal representing a voice from a companion (aperson on the phone) outputted from the connecting terminal 21 of theportable telephone 2 is received by the robot 1 at the connectingterminal 13 and then amplified by the amplifier 14 a. The voice signaloutputted from the amplifier 14 a is converted by the A/D converter 15 ainto a digital signal and supplied to the CPU 16, and after amplified bythe amplifier 14 c, outputted from the speaker 12 c as an audible sound.

A microphone 12 b is disposed to the front of the trunk portion 12 ofthe robot 1. The microphone 12 b is connected with the CPU 16 via anamplifier 14 b and an A/D converter 15 b. The amplifier 14 b isconnected with the connecting terminal 13, so that a signal outputtedfrom the amplifier 14 b is outputted to the portable telephone 2 at theconnecting terminal 13.

A voice of a user in the vicinity of the robot 1 is converted by themicrophone 12 b into an analog voice signal. The voice signal isamplified by the amplifier 14 b. Outputted from the amplifier 14 b, thevoice signal is converted by the A/D converter 15 b into a digitalsignal and supplied to the CPU 16 while outputted to the portabletelephone 2 at the connecting terminal 13.

The A/D converters 15 a and 15 b each perform sampling every 8 ms,thereby converting an analog signal into a digital signal.

The CPU 16 is connected with the plurality of drive circuits 17, 17, . .. 17. The drive circuits 17 are connected with the motors Mrespectively. The motors M are driven by the drive circuits 17.

The CPU 16 is also connected with a voice regeneration circuit 19. A ROMstoring a plurality piece of voice data is incorporated in the voiceregeneration circuit 19. In accordance with a control signal outputtedfrom the CPU 16, the voice regeneration circuit 19 reads the voice datafrom the ROM and outputs a digital voice signal. The voice regenerationcircuit 19 is connected with a D/A converter 19 a. The D/A converter 19a is connected with the speaker 12 c via an amplifier 14 d. A voicesignal outputted from the voice regeneration circuit 19 is converted bythe D/A converter 19 a into an analog signal. After amplified to apredetermined level by the amplifier 14 d, the converted voice signal issupplied to the speaker 12 c and outputted as an audible sound.

A change-over switch lid for manually switching between a speaker modeand a listener mode which will be described later is disposed to a noseportion of the robot 1. A CCD camera 12 f is disposed above themicrophone 12 b of the trunk portion 12. The CCD camera 12 f captures afront image of the robot 1. The CCD camera 12 f is connected with an A/Dconverter 15 c. The A/D converter 15 c is connected with an image memory18. The image memory 18 is connected with the CPU 16.

As the CCD camera 12 f captures the surrounding image, an analog signalrepresenting the image is outputted from the CCD camera 12 f. Thusoutputted analog signal is converted by the A/D converter 15 c into adigital signal and stored in the image memory 18 as digital data of theimage (image data). The stored image data are supplied to the CPU andused for processing which will be described later.

An indicating lamp 12 e for notifying a user of whether it is thespeaker mode or the listener mode is disposed above the CCD camera 12 f.The indicating lamp 12 e is connected with the CPU 16, and theindicating lamp 16 is turned on or off in response to a control signalfrom the CPU 16.

The CPU 16 is connected with a ROM 16 a which stores a program forexecuting the processing which will be described later, and with a RAM16 b which stores data which are temporarily created during execution ofthe program by the CPU 16.

FIG. 4 is an explanatory diagram of functions of the CPU 16.

The CPU 16 comprises a voice processor 120, an image processor 130, anemotion model generator 140, a state managing unit 150 and a replyoperation controller 160. The voice processor 120 comprises a nod timingdetector 121, an input voice volume determining unit 122 and an inputvoice existence determining unit 123. The image processor 130 comprisesa face detector 131, a face motion size detecting unit 132 and adarkness detector 133.

The emotion model generator 140 judges whether the robot is happy, sador angry. The state managing unit 150 determines an interaction state ofthe robot. The reply operation controller 160 determines an action whichcorresponds to a combination of the current emotion of the robotdetermined by the emotion model generator 140 and the currentinteraction state determined by the state managing unit 150. The actiondetermined by the reply operation controller 160 is realized as a signalis fed to the respective drive circuits 17.

The robot 1 can be set to an operation mode. When the operation mode isset to the speaker mode, the robot 1 is controlled so as to behave as ifit were speaking to the user in tune with a voice of the companion inaccordance with the voice signal outputted from the portable telephone2. When the operation mode is set to the listener mode, the robot 1 iscontrolled so as to behave as if it were listening to the user in tunewith the user's voice in accordance with the voice signal outputted fromthe microphone 12 b.

FIGS. 5 and 6 are flow charts showing the sequence of processing in theCPU 16 for setting up of the operation mode of the robot, according tothe first embodiment. The CPU 16 captures the digital signal, namely,voice data outputted from the A/D converter 15 a (Step 101), andcaptures the voice data outputted from the A/D converter 15 b (Step102). The CPU 16 judges whether the digital signals equivalent to 40 ms,i.e., five pieces of the voice data are captured or not (Step 103), andreturns to the step 101 upon failed loading.

When capturing five pieces of the voice data, the CPU 16 calculates anaverage of the five pieces of the voice data representing the companionsupplied at the step 101 (Step 104) and generates binary data from thecalculated average value (Step 105). More specifically, the averagevalue is compared with a predetermined threshold. When the average valueis over the threshold, the binary data has a value of 1. When theaverage value is not over the threshold, the binary data has a value of0. The binary data are stored in an FIFO-type buffer B1 which is capableof storing 1.6 sec-volume of binary data (Step 106). The buffer B1 isformed in a predetermined area within the RAM 16 b.

In a similar manner, an average of the five pieces of the voice dataregarding the user supplied at the step 102 is calculated (Step 107),binary data are generated using the calculated average value and athreshold different from the above threshold (Step 108), and the binarydata are stored in a buffer B2 which is similar to the buffer B1 (Step109).

An MA evaluation value of the companion is calculated from the binarydata within the buffer B1 (Step 110). The MA evaluation value iscalculated as the sum of products is calculated from the binary data anda predetermined group of MA factors based on an MA model.

In a similar manner, an MA evaluation value of the user is calculatedfrom the binary data within the buffer B2 (Step 111). The group of MAfactors used at the step 110 is different from the group of MA factorsused at the step 111.

The CPU 16 thereafter detects the state of the change-over switch 11 dand judges whether the operation mode is manually set (Step 112). Whenthe operation mode is not set manually, the CPU 16 checks whether thereis a signal outputted from the connecting terminal 21 of the portabletelephone 2, i.e., whether there is a signal outputted from the A/Dconverter 15 a, and thus judges whether the portable telephone 2 ishandling a call (Step 113). When the portable telephone 2 is not in acall state, the operation mode is set to the listener mode (Step 114).

When the portable telephone 2 is in a call state, the MA evaluationvalue of the companion is compared with the MA evaluation value of theuser, and the operation mode is set (Step 115). More specifically, thelevel of the MA evaluation value of the companion is compared with thelevel of the MA evaluation value of the user, for example. When the MAevaluation value of the companion is equal to or larger than the MAevaluation value of the user, the CPU 16 invokes the speaker mode. Whenthe MA evaluation value of the companion is smaller than the MAevaluation value of the user, the CPU 16 invokes the listener mode. Theprocedure then returns to the step 101.

FIG. 7 is a flow chart showing the sequence of processing in the CPU 16for control of actions by the robot 1 according to the first embodiment.The CPU 16 judges to which mode the operation mode is currently set(Step 121). When the operation mode is set to the speaker mode, based onthe MA evaluation value of the companion calculated through theprocessing shown in the flow charts in FIGS. 5 and 6, the CPU 16 selectsone of the plurality of operation patterns of the robot 1 (Step 122),and proceeds to the step 124. During the processing at the step 122, theCPU compares the MA evaluation value of the companion with a pluralityof thresholds and selects the operation patterns associated with therespective thresholds. For instance, when the MA evaluation value of thecompanion is over the threshold which is for activating a blinkingaction, the CPU selects the operation pattern that the eyelids 11 cswing in the up/down direction. Further, when the MA evaluation value ofthe companion is over the threshold for opening and closing the mouth 11a which is larger than the threshold for the blinking action, forexample, the CPU selects the operation pattern that the mouth 11 a opensand closes.

At the step 121, when the operation mode is currently set to thelistener mode, based on the MA evaluation value of the user, one of theplurality of operation patterns of the robot 1 is selected (Step 123),and the procedure proceeds to the step 124. In processing at the step123, the MA evaluation value of the user is compared with a plurality ofthresholds and the operation patterns associated with the respectivethresholds are selected. For instance, when the MA evaluation value ofthe user is over the threshold which is for activating a nodding action,the operation pattern that the head portion 11 sways in the up/downdirection is selected. Meanwhile, when the MA evaluation value of theuser is beyond the threshold which is for activating an agreeing actionwhich is larger than the threshold for the nodding action, the operationpattern that the mouth 11 a opens and closes and a voice saying, “Yeah,yeah,” comes out is selected.

Next, a control signal which calls for an action corresponding to theselected operation pattern is outputted to the drive circuits 17 or thevoice regeneration circuit 19 (Step 124). During this processing, whenthe operation pattern that the mouth 11 a opens and closes is selectedfor instance, a control signal which instructs the mouth 11 a to openand close is supplied to the drive circuit 17 which drives the motor Mwhich is linked with an action mechanism of the mouth 11 a. Meanwhile,when the operation pattern for the agreeing action is selected forexample, a control signal which instructs the mouth 11 a to open andclose is supplied to the drive circuit 17 and a control signal whichdemands outputting of a voice saying, “Yeah, yeah,” is supplied to thevoice regeneration circuit 19.

The CPU 16 reads the image data acquired by the CCD camera 12 f out ofthe image memory 18 (Step 125). As the image data are treated with imageprocessing, the CPU 16 detects the position of the user within the image(Step 126). Based on the detected position of the user, the CPU 16outputs a control signal to the drive circuits 17, 17 which drive themotors M, M which are linked to an action mechanism of the wheelsdisposed in a lower section of the robot 1 such that the robot 1 movesto face the user (Step 127), and returns to the step 121.

The processing shown in the flow charts in FIGS. 5 and 6 and theprocessing shown in the flow chart in FIG. 7 described above areexecuted in a predetermined cycle through multi-task processing by theCPU 16.

Operations of the respective portions of the CPU which determine theactions of the robot will now be described in detail.

FIG. 8 is a flow chart showing the sequence of processing in the nodtiming detector 121.

First, the analog voice signal supplied from the microphone 12 b is A/Dconverted in a cycle of 8 msec (Step 131). Every time five conversionsare made, i.e., for every cycle of 40 ms (Step 132), an average value ofAD conversion values over the five conversions is calculated (Step 133).As thus obtained average value is threshold processed using an ON/OFFthreshold, a time-series ON/OFF pattern over the past 1.6 sec isgenerated (Step 134).

Using a linear prediction model regarding time-series ON/OFF patterns,the nod timing detector 121 determines whether it is the timing to nod.When it is the timing to nod, the nod timing detect or 121 sends amessage indicative of this to the emotion model generator 140 (Step135). The timing to nod is the timing which is appropriate to respond towords spoken by the companion.

Through the sequence of processing described above, the nod timingdetector 121 judges whether it is the timing to nod for every 40 ms, andwhen judging that it is the nod timing, the nod timing detector 121sends a message indicating that it is the nod timing to the emotionmodel generator 140.

Using three thresholds, for every 40 ms, the input voice volumedetermining unit 122 threshold processes the average value of the ADconversion values obtained at the step 133. As a result, the volume ofthe input voice is classified into four types of high, middle, low andnone. A discriminant signal corresponding to the high level, the middlelevel or the low level of the input voice is outputted from the inputvoice volume determining unit 122 and sent to the emotion modelgenerator 140.

From ON/OFF patterns obtained at the step 134, for every 1.6 sec, theinput voice existence determining unit 123 calculates the proportion ofON patterns. When the proportion of the ON patterns is larger than apredetermined value (0.5 for instance), the discriminant signalindicative of that there is an input voice is outputted. When theproportion of the ON patterns is equal to or smaller than apredetermined value, the discriminant signal indicative of that there isno input voice is outputted. The discriminant signal is sent to thestate managing unit 150.

Based on the input image data from the CCD camera 12 f, the facedetector 131 extracts a skin color area. Next, based on the size and theshape of the extracted skin area, whether there is a face area in theinput image is determined. A message which is indicative of the resultis transmitted to the emotion model generator 140 and the state managingunit 150. The face detection by the face detector 131 is performed in apredetermined cycle.

When the face detector 131 detects the face area, the face motion sizedetecting unit 132 detects the size of a motion of the face area tothereby determine the size of the motion of the face. A message which isindicative of the result is transmitted to the emotion model generator140. The detection of the size of the motion of the face by the facemotion size detecting unit 132 is performed in a predetermined cycle.

The darkness detector 133 detects surrounding darkness based on theinput image data from the CCD camera 12 f. Detecting that it is darkaround, the darkness detector 133 sends a message indicative of this tothe state managing unit 150. The darkness detection by the darknessdetector 133 is performed in a predetermined cycle.

On the assumption that the robot has three emotions of happy, sad andangry feelings, operations of the emotion model generator 140 will nowbe described. The emotion model generator 140 has parameters(hereinafter referred to as “emotion parameters”) Level_(Hap),Level_(Sad) and Level_(Ang) which express the intensity of happy, sadand angry feelings.

Each emotion parameter repeatedly changes due to a few factors(stimulation provided from outside) which change the emotions. Thelargest one of the three emotion parameters becomes the current emotion.When the three emotion parameters are the same, it is determined thatthe robot is happy.

The factors which change the emotions are the nod timing detected by thenod timing detector 121, the volume of the input voice determined by aninput voice volume determining unit 122, a face detected by the facedetector 131 and the size of the motion of the face detected by the facemotion size detecting unit 132. The nod timing serves as an indexshowing whether the conversation is lively. It is interpreted that moreoften the nod timing is detected, livelier the conversation is.

When the volume of the input voice is approximately at the middle leveland the face of a companion is detected for instance, since the robot isbeing spoken to merrily and can see the face of the companion, theemotion parameter Level_(Hap) expressing the degree of happinessincreases. In this case, the robot becomes happy.

On the other hand, when the volume of the input voice is at the lowlevel and the face of a companion is not detected, since the robot isbeing spoken to desolately and can not see the face of the companion,the emotion parameter Level_(Sad) expressing the degree of sorrowincreases. In this case, the robot becomes sad. Meanwhile, when thevolume of the input voice is at the high level, since the robot is beingyelled at, the emotion parameter Level_(Ang) expressing the degree ofanger increases. In this case, the robot becomes angry.

FIG. 9 is a flow chart showing the sequence of processing in the emotionmodel generator 140.

Based on the factors which change the emotions, the emotion modelgenerator 140 controls the emotion parameter Level_(Hap) expressing thedegree of happiness, the emotion parameter Level_(Sad) expressing thedegree of sorrow and the emotion parameter Level_(Ang) expressing thedegree of anger.

First, the emotion model generator 140 initializes the respectiveemotion parameters Level_(Hap), Level_(Sad) and Level_(Ang) (Step 141).The respective emotion parameters Level_(Hap), Level_(Sad) andLevel_(Ang) have a value from 0 to 1000. In the initialization at thestep 141, the respective emotion parameters Level_(Hap), Level_(Sad) andLevel_(Ang) are set to 500 (neutral value).

Next, the emotion model generator 140 waits for receipt of a messagefrom the nod timing detector 121, the input voice volume determiningunit 122, the face detector 31 or the face motion size detecting unit132 (Step 142). Receiving a message transmitted from either of theseunits, the emotion model generator 140 parses the received signal (Step143) and calculates the respective emotion parameters (Step 144).

The emotion model generator 140 sends to the reply operation controller160 the emotion corresponding to the largest emotion parameter among thecalculated emotion parameters as the present emotion of the robot (Step145). When the emotion parameters are all equal, the emotion modelgenerator 140 sends the happy emotion to the reply operation controller160 as the present emotion of the robot. After this, the emotion modelgenerator 140 returns to the step 142. The respective emotion parametersare calculated based on the formulas below:Point_(Ctl)=Gain_(All)*Point_(Local)Level_(Hap)(t)=Gain_(Hap)*Point_(Ctl)+Level_(Hap)(t−1)Level_(Sad)(t)=Gain_(Sad)*Point_(Ctl)+Level_(Sad)(t−1)Level_(Ang)(t)=Gain_(Ang)*Point_(Ctl)+Level_(Ang)(t−1)

In the formula, Gain_(All) is an overall gain which may be 1, forinstance. Point_(Local) is a point which is set in advance for each oneof the factors which change the emotions. Gain_(Hap), Gain_(Sad) andGain_(Ang) are gains which are set in advance for the respective factorswhich change the emotions.

Point_(Local), Gain_(Hap), Gain_(Sad) and Gain_(Ang) are set as shown inTable 1. Point_(Local) is determined experientially from the frequencyof occurrence of the factors which change the emotions and theimportance levels of the factors which change the emotions. Gain_(Hap),Gain_(Sad) and Gain_(Ang) are positive values with respect to thefactors which increase the corresponding emotion parameters but areotherwise negative values.

TABLE 1 Point_(Local) Gain_(Hap) Gain_(Sad) Gain_(Ang) NOD TIMING 10 +1−1 −1 VOICE HIGH 1 −1 −1 +1 LEVEL MIDDLE +1 −1 −1 LOW −1 +1 −1 FACEDETECTED 5 +1 −1 −1 NOT DETECTED −1 +1 −1 FACE DETECTED 2 +1 −1 −1MOTION NOT DETECTED −1 +1 −1

Where the respective emotion parameters Level_(Hap), Level_(Sad) andLevel_(Ang) are set to 500 for instance, when a message indicative ofdetection of a face is received from the face detector 131,Point_(Local)=5, Gain_(Hap)=+1, Gain_(sad)=−1 and Gain_(Ang)=−1. Hence,Point_(Ctl)=5, Level_(Hap)=505, Level_(Sad)=495 and Level_(Ang)=495. Asa result, the emotion parameter Level_(Hap) expressing the degree ofhappiness increases and the emotion parameters Gain_(Sad) and Gain_(Ang)expressing the degree of sorrow and anger decrease.

Based on thinking that each emotion is supposed to tranquilize whenthere is a continuing state that a message is not received, it isdesirable to control each emotion parameter so that each emotionparameter becomes close to the neutral value (500).

Although the factors which change the emotions are the nod timingdetected by the nod timing detector 121, the volume of the input voicedetermined by an input voice volume determining unit 122, a facedetected by the face detector 131 and the size of the motion of the facedetected by the face motion size detecting unit 132 in the embodimentdescribed above, any combinations of these may be used as the factorswhich change the emotions.

Operations of the state managing unit 150 will now be described.Considering a message from the input voice existence determining unit123, a message from the face detector 131 and a message from thedarkness detector 133 as events, the state managing unit 150 causes theinteraction state to transit in accordance with the state transitiontable in Table 2, to thereby determine the interaction state of therobot. Thus determined current interaction state of the robot istransmitted to the reply operation controller 160.

TABLE 2 INTERACTION STATE In Conversation Autonomic EVENT ConversationInterruption Action Sleep Input No Transi- → In → In → In Voice tionConversation Conversation Conversation No → Conver- No Transition NoTransition No Transition Input sation Voice Interruption No No Transi- →Autonomic No Transition No Transition Speaker tion Action Dark- NoTransi- → Sleep → Sleep No Transition ness tion

The interaction state of the robot can be four types of “InConversation,” “Conversation Interruption,” “Autonomic Action” and“Sleep” in this example.

The events include an event which indicates that there is an input voicewhich is detected by the input voice existence determining unit 123, anevent which indicates that there is no input voice which is detected bythe input voice existence determining unit 123, an event which indicatesthat there is no speaker which is detected by the face detector 131(i.e., no face detected) and an event which indicates that it is darkwhich is detected by the darkness detector 133.

For example, when the interaction state is “In Conversation”, uponoccurrence of an event which indicates that there is no input voice, theinteraction state transits to “Conversation Interruption”. In thisstate, as an event which indicates that there is no speaker occurs, theinteraction state transits to “Autonomic Action”.

When the interaction state is “Conversation Interruption” or “AutonomicAction”, upon occurrence of an event which indicates that it is dark,the interaction state transits to “Sleep”. In this state, as an eventwhich indicates that there is an input voice occurs, the interactionstate transits to “In Conversation”.

The events may include at least an event which indicates that there isan input voice which is detected by the input voice existencedetermining unit 123, an event which indicates that there is no inputvoice which is detected by the input voice existence determining unit123, and an event which indicates that there is no speaker which isdetected by the face detector 131.

Operations of the reply operation controller 160 will now be described.Based on the reply operation table shown in FIG. 10, the reply operationcontroller 160 determines an operation which corresponds to acombination of the current emotion of the robot determined by theemotion model generator 140 and the current interaction state determinedby the state managing unit 150. The robot acts based on thus determinedoperation. Although not shown in FIG. 10, in practice, a voice matchingwith the emotion, i.e., a happy sound voice for a happy feeling, a sadsound voice for a sad feeling or an angry sound voice for an angryfeeling, is outputted from the speaker 12 c.

Although the reply operation controller 160 determines an action of therobot in accordance with a combination of the current emotion of therobot determined by the emotion model generator 140 and the currentinteraction state determined by the state managing unit 150 in theembodiment described above, the present invention is not limited tothis. An action of the robot may be determined considering the characterof the robot as well.

When the character of the robot is to be considered, a character settingunit for setting up the personality of the robot is disposed. Thecompanion sets up the character of the robot according to his or herpreference. As the types of the character, “timid,” “hot-tempered,”“gentle,” “cheerful,” “peaceful” and the like are prepared. For each oneof the character types, a reply operation table as that shown in FIG. 10is generated. An action of the robot is determined using the replyoperation table which corresponds to the character type which is set.

When the character type is set to “hot-tempered” and there is atransition to “Autonomic Action” since there is no speaker for instance,the robot acts roughly with surge of an “angry” feeling, e.g.,“STRUGGLE”s.

When the character type is set to “peaceful” and there is a transitionto “Autonomic Action” since there is no speaker, the robot expresses theangry feeling inside itself by “heaving” etc. with surge of an “angry”feeling.

While the foregoing regarding the respective units has described theoperations based on the voice signal inputted through the microphone 12b, control using an input signal from a communication signal is alsopossible according to the present invention.

An automatic distribution/receipt system will now be described whichpermits automatic downloading of an operation program registered in anexternal server apparatus, such as a web page of a company, to therobot. FIG. 11 is a schematic diagram showing an essential structure ofan automatic distribution/receipt system for a robot operation programaccording to the present invention.

In FIG. 11, the automatic distribution/receipt system includes the robot1, the portable telephone 2 which is connected with the robot 1 in amanner described above, a communication network (Internet line) 24containing a fixed line network 22 which is connected with the portabletelephone 2 via a base station, and an external server apparatus 23which can link to the communication network 24 and has a database inwhich various types of service information such as web pages ofcompanies are registered.

Client software called a browser for browsing through web assets isusually installed in the portable telephone 2. The user of the portabletelephone 2 accesses the database of the external server apparatus 23,using this browser.

The external server apparatus 23 such as a web page of a company whichis a link destination is equipped in advance with a program (Aplet)written in Java for downloading of an operation program. As an access ismade from the portable telephone 2 to the enterprise web page throughthe fixed line network 22 which is a communication network, the programabove (Aplet) is executed automatically, whereby an operation programfor robots is downloaded to the robot 1 via the portable telephone 2.

Interpreting the downloaded operation program, the robot 1 drives andcontrols the motors M.

The flow of the automatic downloading of the operation program usingsuch a structure above will be described based on the flow chart in FIG.12.

First, the communication terminal checks whether there is an access fromthe portable telephone 2 on the Internet line 24 to the external serverapparatus 23 such as a homepage of web pages of a company (Step 201).When there is no access, the procedure returns to the start. When thereis an access, whether an Aplet which is programmed in Java so as toautomatically download a CM-operation program for the robot 1 exists inthe company web pages 23 is detected (Step 202). When there is no Aplet,the procedure returns to the start. When there is an Aplet existing, theAplet (Java/program) is downloaded automatically (Step 203).

Following this, the Aplet is executed automatically and the procedurereturns to the start (Step 204). The CM-operation program for the robot1 thus downloaded to the portable telephone 2 is downloadedautomatically to the robot 1 through infrared data communication (IrDA)(Step 205).

Owing to the operation program existing in the robot 1 in advanceusually in the autonomic mode, the robot is in an autonomic operationstate which allows to freely act (Step 206). Through the automaticdownloading by means of the Aplet described above, whether the robot 1has received the new operation program for robot is checked (Step 207).When the new operation program has not been received, the procedurereturns to the start.

When the new operation program has been received, the new CM-operationprogram received at the portable telephone 2 is set up (stored) on amemory (ROM such as a f lash memory) (Step 208). Further, if thus setCM-operation program satisfies an operation condition such as time (Step209), the content of the CM-operation program is interpreted, convertedinto information necessary to actually make a move and outputted to thedrive circuits (Step 210).

With execution of the CM-operation program, the robot 1 acts in practice(Step 211) and the procedure returns to the start. When the operationcondition is not satisfied at the step 209, the procedure returns to thestart.

Processing by the external server apparatus is shown in the flow chartin FIG. 13.

The external server apparatus checks whether there is an access from theportable telephone 2 to a homepage of web pages of a company or the likewhich is the external server apparatus 23 (Step 212). When there is noaccess, the procedure returns to the start. When there is an access, webpage data containing an Aplet are downloaded (Step 213) and theprocedure returns to the start.

In accordance with the flow of the operations described above, theCM-operation program for robots on the external server apparatus 23 suchas a web page of a company is automatically downloaded to the robot 1.The downloaded operation program is temporarily stored in a storage part(memory) which is provided in the robot 1, so that it is possible toexecute the operation program in accordance with a condition or in realtime simultaneously with downloading.

The configuration of the hardware shown in FIG. 3 will now be describedwith reference to the operation flow chart in FIG. 14.

First, the CPU 16 determines whether it is necessary to change theconfiguration of the hardware on the memory 31 which is mounted to themain control board 25 (Step 221). When a change is necessary, the mostupdate data are downloaded onto the memory 31 via the CPU 16 and fromthe communication terminal 27 through various types of the communicationinterfaces 20 (Step 222). With this data downloaded to the non-volatilememory described above such as a flash memory, unless the hardware needsbe changed the next time the power source is turned on, it is possibleto configure the FPGA 29 using this data without downloading(corresponding to when there is no change requested at the step 221).

As the downloading completes or preparation to configure the FPGA 29using the data stored in advance in the non-volatile memory completes atthe step 222, configuring is executed starting with the FPGA 29 on themain control board 25 using the dedicated control signal bus 28 (Step223).

As the configuring completes and the hardware structure is accordinglydecided at the step 223, software dependent upon the hardware isinitialized in a similar manner to that regarding normal equipment usingmicrocomputers (Step 224). A program of application softwarecorresponding to the hardware is then executed (Step 225).

It is not always necessary that there is one piece of configuration datafor FPGAs. A plurality piece of configuration data may be in thenon-volatile memory. In addition, data for configuring may be usedselectively.

The hardware can be modified at the time of execution of a program ofapplication software. For this purpose, when there is a request forchange on the application for instance (Step 226), the procedure returnsto the step 222 and different data are downloaded, or alternatively,configuring is performed using different data on the memory 31.

The operation flow chart which describes configuring of the hardware inFPGAs 26 mounted on the outer connection boards 32 is similar to FIG.14, and therefore, will not be described.

By means of the structure above, when the portable telephone 2 ishandling a call, the robot 1 moves so as to face the user and opens andcloses the mouth 11 a, nods or otherwise acts in tune with the voice ofthe person on the phone, thereby behaving as if it were in conversationwith the user. When the portable telephone 2 is not handling a call, therobot 1 moves so as to face the user and nods, agrees or otherwise actsin tune with the voice of the user, thereby behaving as if it were inconversation with the user. With an operation program for robots on theexternal server apparatus 23 such as a web page of a company downloaded,the robot 1 can perform totally new movements.

SECOND EMBODIMENT

FIG. 15 is a schematic diagram showing an essential structure of asecond embodiment of a robot according to the present invention. A robotsystem according to the second embodiment comprises a robot 1, acomputer 3 which functions as a communication apparatus and a portabletelephone 4. The robot 1 comprises a radio-communication unit 110 whichhas a communication function such as PHS data communication andBluetooth.

FIG. 16 is a block diagram showing a structure of the robot 1 accordingto the second embodiment. The robot 1 outputs a control signal from aCPU 16 to the drive circuits 17 in response to command data received atthe radio-communication unit 110. This enables to remotely operate therobot 1. FIG. 17 shows one example of the command data for the robot 1.The command data are formed by text commands of ASCII codes andparameters which express the amount of motion, a traveling speed, arotational angle, a rotation angle speed, etc. In the command data “MWSB20 10” for example, “MWSB” is a text command which expresses an actionthat the wheels of the robot 1 are driven to retract, “20” is aparameter which expresses the amount of motion, and “10” is a parameterwhich expresses the traveling speed. Hence, the command data instruct toback off over the distance of 20 at the speed of 10.

In FIG. 17, to WALK is to have the robot 1 act as if it were walking bycausing the wheels disposed to leg portions of the robot 1 to rotate andstop repeatedly in predetermined cycles.

As a plurality pieces of command data are linked with “:”, and “;” isadded after the last piece of command data, an operation program callingfor a multiple of actions is created. An operation program “WSB 2010:WW:D 100:MNR 10 10”, for instance, will now be described.

This operation program expresses an action that after the robot retractsover the distance of 20 at the speed of 10, the robot stops for a periodof 100 and thereafter tilts the head for the amount of motion of 10 atthe speed of 10.

The other structures of the robot 1 according to the second embodimentare similar to those of the robot 1 according to the first embodiment,and therefore, will be denoted at the same reference symbols but willnot be described.

FIG. 18 is a block diagram showing a structure of the computer 3. Thecomputer 3 comprises a CPU 31, a ROM 32 which stores a program to beexecuted by the CPU 31, data used by this program, etc., a RAM 33 whichstores data which are temporarily generated during execution of theprogram by the CPU 31, a hard disk apparatus 34 which stores a programto be executed by the CPU 31, data used by this program, etc., and aninput and output interface 35. Connected with the input and outputinterface 35 are a radio-communication unit 35 a which can communicatewith the radio-communication unit 110 of the robot 1, a modem 35 b whichis connected with a telephone line, a display 35 c such as a CRT and anLCD, and an input apparatus 35 d such as a key board and a mouse.

Through the modem 35 b, the computer 3 can dial-up connect with acomputer 5 of an Internet service provider or the like. Hence, thecomputer 3 can connect with the Internet 6. In the computer 3, a WWWserver apparatus program is stored in the hard disk apparatus 34. Withthe WWW server apparatus program downloaded to the RAM 33, in responseto a request from other computer, a portable telephone or the like whichis connected on the Internet 6 and in a process of executing the WWWclient program, the computer 3 can send HTML data, image data or thelike stored in the hard disk apparatus 34 to the computer or theportable telephone.

The portable telephone 4 incorporates a CPU, a ROM, a RAM and the like(not shown). The portable telephone 4 can connect with the Internet 6via a computer 7 of a telecommunications carrier. A WWW client programis stored in the ROM. As the CPU executes the WWW client program, theportable telephone 4 can request a computer which is connected on theInternet 6 for HTML data, image data or the like, receive the requestedHTML data, image data or the like and display a screen imagecorresponding to the received data on an LCD.

FIGS. 19, 20, 21 and 22 are flow charts showing the flow of operationsof the robot system according to the second embodiment. Wishing tomonitor the state inside the house, a user manipulates the portabletelephone 4 and calls the computer 3. The portable telephone 4,accepting the manipulations given from the user, sends a call signal tothe computer 3 (Step 301). The computer 3 receives the call signal fromthe portable telephone 4 through the modem 35 b (Step 302), and starts acall (Step 303). Following this, the computer 3 sends a voice signalrepresenting a voice saying, “Connecting with the Internet. Please waitfor transmission of a reply e-mail.” to the portable telephone 4 (Step304). The portable telephone 4 receives the voice signal (Step 305), andoutputs a voice from a built-in speaker. The computer 3 then ends thecall (Step 306).

Next, the computer 3 sends the computer 5 a connection request signal,and makes a connection with the Internet 6 (Step 307). Staying connectedwith the Internet 6, the computer 3 sends an e-mail (an electronic mail)formed by text data containing URL which denotes the memory location ofthe HTML data in the computer 3 to an e-mail address which is associatedwith the portable telephone 4 (Step 308).

Although the second embodiment requires to send the e-mail when theportable telephone 4 calls up the computer 3, this is not limiting. Forinstance, the robot 1 may perform image processing on an image capturedby the CCD camera 12 f to thereby detect abnormality such as intrusioninto the house. Upon detection of such abnormality, the robot 1 mayrequest the computer 3 to send the e-mail so that the computer 3,receiving the e-mail, sends the e-mail.

The operations of the robot 1 are usually controlled so as toautonomously act. To be more specific, the robot 1 performs imageprocessing on an image captured by the CCD camera 12 f to thereby detectthe captured human being or the like. Detecting no human beings or thelike within a predetermined period of time, the robot 1 switches to thestand-by state to wait for the receipt of a signal from the computer 3.

The computer 3 sends the robot 1 an image data request signal whichrepresents a request for transmission of image data (Step 309). Therobot 1 receives the image data request signal (Step 310), and the CCDcamera 12 f obtains image data (Step 311). The robot 1 thereafter sendsthus obtained image data to the computer 3 (Step 312).

The computer 3 receives the image data (Step 313), converts the imagedata into image data in a format which can be displayed by the WWWclient program of the portable telephone 4 (e.g., JPEG or GIFF) andstores the image data in the hard disk apparatus 34 (Step 314).

The portable telephone 4 receives an e-mail (Step 315). As a usermanipulates the portable telephone 4 and selects the URL contained inthe e-mail, the portable telephone 4 sends the computer 3 an HTML datarequest signal which represents a request for transmission of HTML data(Step 316). The computer 3 receives the HTML data request signal (Step317). When the HTML data request signal is not received within apredetermined period of time, the connection with the computer 5 ends(Step 318) and the procedure ends.

When the HTML data request signal is received within the predeterminedperiod of time, the computer 3 sends to the portable telephone 4 theHTML data stored in the hard disk apparatus 34 and the image data storedat the location within the hard disk apparatus 34 which is expressed bythis HTML data (Step 319).

The portable telephone 4 receives the HTML data and the image data (Step320), and displays a screen image corresponding to the received data onthe LCD (Step 321). The screen image shows a plurality of buttons forinstructing the robot 1 to act. For instance, as the user manipulatesthe portable telephone 4, selects one of the buttons and instructs topress this button, operation instruction data which demand the robot 1to rotate to the right-hand side are transmitted to the computer 3. Whenthe user provides the instruction calling for such an action by therobot 1, the portable telephone 4 sends the operation instruction datawhich express this instruction regarding action to the computer 3 (Step322).

The computer 3 receives the operation instruction data from the portabletelephone 4 (Step 323), converts the operation instruction data intocommand data or operation program as that described above (Step 324),and sends this data to the robot 1 (Step 325). Receiving the data (Step326), the robot 1 is operated in accordance with the received data (Step327). Upon an end of the operation, the robot 1 sends an operation endsignal expressing the end of the operation to the computer 3 (Step 328).The computer 3 receives the operation end signal (Step 329). Thecomputer 3 sends to the portable telephone 4 operation end dataindicative of the end of the operation by the robot 1 (Step 330).Receiving the operation end data (Step 331), the portable telephone 4returns to the step 321 and displays a screen image.

When the computer 3 does not receive the operation instruction datawithin the predetermined period of time at the step 323, the computer 3sends the image data request signal to the robot 1 (Step 332). The robot1 receives the image data request signal (Step 333), obtains image dataat the CCD camera 12 f (Step 334), and sends thus obtained image data tothe computer 3 (Step 335).

Receiving the image data (Step 336), the computer 3 converts the imagedata into image data in the afore-mentioned format and stores the imagedata in the hard disk apparatus 34 (Step 337), and sends the image datato the portable telephone 4 (Step 338). The portable telephone 4receives the image data (Step 339), and updates the picture on the LCDinto a screen image which corresponds to the image data (Step 340).

When wishing to end the connection with the computer 3, the usermanipulates the portable telephone 4 and causes the portable telephone 4to send a connection end signal which expresses an end of connection(Step 341). Upon the receipt of the connection end signal (Step 342),the computer 3 ends the connection (Step 343), and the procedure ends.When the computer 3 does not receive the connection end signal, thecomputer 3 returns to the step 323.

Although the second embodiment requires to log on the computer 3 to theInternet 6, causes the portable telephone 4 to remotely manipulate therobot 1 through the computer 3 and have the robot 1 send image data orthe like to the portable telephone 4, this is not limiting.Alternatively, the robot 1 may incorporate a modem and a WWW serverapparatus program may be stored in the ROM 16 a of the robot 1 so thatwhen the robot 1 is connected to the Internet 6 via the modem, theportable telephone 4 remotely manipulates the robot 1, or that imagedata obtained by the CCD camera 12 f are stored in the RAM 16 b of therobot 1 and downloaded to the portable telephone 4.

By means of the structure described above, when away from home leaving achild, an old person or the like who needs be taken care of at home, theuser can learn about the state inside the house by remotely manipulatingthe robot 1 using the portable telephone 4 from a remote place andconfirming on the portable telephone 4 an image captured by the CCDcamera 12 f of the robot 1.

As the robot 1 which is capable of autonomously acting monitors aroundin the house, the child or old person left at home can develop afriendly feeling toward the robot 1 and therefore feel more secured ascompared to where a monitor camera monitors around in the house.

THIRD EMBODIMENT

FIGS. 23 and 24 are a perspective view and a block diagram showing anessential structure of the third embodiment of the robot according tothe present invention. Portions which are similar to those in thestructure of the robot 1 according to the first embodiment will bedenoted at the same reference symbols but will not be described.

An infrared rays sensor 12 d is disposed to the side of the CCD camera12 f in a predetermined distance. The infrared rays sensor 12 d isconnected with an A/D converter 15 d, and the A/D converter 15 d isconnected with the CPU 16. The infrared rays sensor 12 d outputs ananalog signal which corresponds to the quantity of infrared light aroundthe infrared rays sensor 12 d, and the analog signal is converted by theA/D converter 15 d into a digital signal and fed to the CPU 16.

In FIG. 24, denoted at 110 is a radio-communication unit which has acommunication function such as PHS data communication, Bluetooth or thelike. The portable telephone 2 is capable of storing in a built-in RAM(not shown) the voice levels at which the user talked during calls, thehistory of calls which the user has received and the history of callswhich the user has made in one day (everyday life information). Further,a CCD camera (not shown) is incorporated in the portable telephone 2. Asthe user manipulates the portable telephone 2, the CCD camera capturesthe face of the user. A CPU (not shown) mounted in the portabletelephone 2 processes the captured image, and therefore, it is possibleto evaluate the user's complexion among the three levels of good, normaland poor, and store the information regarding the evaluation result(physiological information) in the RAM.

Comprising a radio-communication unit (not shown) which can communicatewith the radio-communication unit 110, the portable telephone 2 can sendto the robot 1 the information regarding the stored voice levels, thehistory of received calls, the history of made calls and the user'scomplexion.

In FIG. 24, denoted at 8 is a steps counter which counts the number ofsteps the user walked. The steps counter 8 stores information regardingthe measured number of steps in one day (everyday life information).Further, the steps counter 8 comprises a radio-communication unit (notshown) which can communicate with the radio-communication unit 110, andtherefore, can transmit to the robot 1 the stored information regardingthe number of steps.

In FIG. 24, denoted at 9 is a wearable sensor which the user can wear onhis or her body. The wearable sensor 9 is capable of measuring theuser's heart rate (physiological quantity). As the heart rate ismeasured, a CPU (not shown) incorporated in the wearable sensor 9determines whether the heart rate is normal. More specifically, therange of the normal heart rate is defined in advance. When the measuredheart rate is within this range, it is judged that the heart rate isnormal. When the measured heart rate is not within the range, it isjudged that the heart rate is abnormal. The wearable sensor 9 as wellcomprises a radio-communication unit (not shown) which can communicatewith the radio-communication unit 110, and when it is judged that theheart rate is abnormal, data which are indicative of the abnormality inheart rate (physiological information) is transmitted to the robot 1.

The CPU 16 of the robot 1 is capable of detecting that the portabletelephone 2 or the steps counter 8 comes into the communication area ofthe radio-communication unit 110.

Map data depicting inside a building in which the robot 1 is used arestored in the RAM 16 b of the robot 1 in advance. Disposed to a lowerpart of the trunk portion 12 is a rotary encoder (not shown) whichdetects the rotations of the wheels which the robot 1 use to travel.Since the CPU 16 can detect the distance and direction of travelingbased on an output from the rotary encoder, the CPU 16 can detect thecurrent location of the robot 1 based on the distance and direction oftraveling and the map data.

FIG. 25 is a flow chart showing the sequence of processing in the CPU 16of the third embodiment of the robot according to the present invention.First, the CPU 16 judges whether the portable telephone 2 or the stepscounter 8 exists in the communication area of the radio-communicationunit 110 (Step 401). When the portable telephone 2 comes into thecommunication area, first control processing which will be describedlater is performed (Step 402). When the steps counter 8 comes into thecommunication area, second control processing which will be describedlater is performed (Step 403). When neither the portable telephone 2 northe steps counter 8 exists in the communication range, the CPU judgeswhether the data which are indicative of an abnormal heart rate arereceived by the radio-communication unit 110 from the wearable sensor 9(Step 404). When there is no receipt of the data, the CPU 16 returns tothe step 401. When there is receipt of this data, third controlprocessing which will be described later is performed (Step 405) and theprocedure ends.

FIG. 26 is a flow chart showing the sequence of the first controlprocessing by the CPU 16. The CPU 16 causes the radio-communication unit110 to send a signal requesting to send information to the portabletelephone 2 (Step 411). When receiving the signal, the portabletelephone 2 sends to the robot 1 the information regarding the voicelevels, the history of calls which the user has received, the history ofcalls which the user has made and the user's complexion. After theradio-communication unit 110 receives the information, the informationis received by the CPU 16 (Step 412).

Next, a counter C showing the activity level of the user in one day isset to 0 (Step 413). The CPU 16 then determines the volume of the user'svoice during a call, from the voice level contained in the receivedinformation. The counter C is updated in accordance with the judgment(Step 414). More specifically, the range of a voice level for a normalvolume of voices is defined in advance. When the voice level containedin the received information is within the range, judging that the volumelevel of the voice is normal, the CPU 16 does not change the counter C.When the voice level contained in the received information is over therange, judging that the voice is loud, the CPU 16 adds 1 to the counterC. When the voice level contained in the received information is belowthe range, judging that the voice is small, the CPU 16 subtracts 1 fromthe counter C.

In addition, from the history of calls which the user has receivedcontained in the received information, the CPU 16 determines whether thecalls received at the portable telephone 2 in one day are many. The CPU16 updates the counter C in accordance with the determination (Step415). More specifically, the range of the number of received calls in anormal condition is defined in advance. The number of the received callsis obtained from the history of the received calls contained in thereceived information. When the number of the received calls is withinthe range, judging that the number of the received calls is a normalnumber, the CPU 16 does not change the counter C. When the number of thereceived calls is over the range, judging that the number of thereceived calls is a large number, the CPU 16 adds 1 to the counter C.When the number of the received calls is below the range, judging thatthe number of the received calls is a small number, the CPU 16 subtracts1 from the counter C.

In a similar manner, from the history of calls which the user has madecontained in the received information, the CPU 16 determines the numberof the made calls from the portable telephone 2 in one day. The CPU 16updates the counter C in accordance with the determination (Step 416).More specifically, the range of the number of made calls in a normalcondition is defined in advance. The number of the made calls isobtained from the history of the made calls contained in the receivedinformation. When the number of the made calls is within the range,judging that the number of the made calls is a normal number, the CPU 16does not change the counter C. When the number of the made calls is overthe range, judging that the number of the made calls is a large number,the CPU 16 adds 1 to the counter C. When the number of the made calls isbelow the range, judging that the number of the made calls is a smallnumber, the CPU 16 subtracts 1 from the counter C.

Further, the CPU 16 updates the counter C based on the informationregarding the user's complexion, or facial color, contained in thereceived information (Step 417). More specifically, when the user'scomplexion information contained in the received information expresses aresult evaluating that the complexion is normal, the CPU 16 does notchange the counter C. When the user's complexion information expresses aresult evaluating that the complexion is good, the CPU 16 adds 1 to thecounter C. When the user's complexion information expresses a resultevaluating that the complexion is poor, the CPU 16 subtracts 1 from thecounter C.

The CPU 16 judges the count of the counter C (Step 418). When C>0, theCPU 16 determines that the user has spent the day actively. In thiscase, the CPU 16 outputs a control signal to the voice regenerationcircuit 19 and the drive circuits 17 so that the robot 1 outputs a voicesaying, “You had a good day,” and behaves as if it were dancing (Step419), and returns.

When C=0 at the step 418, the CPU 16 judges that the user has spent theday normally. In this case, the CPU 16 outputs a control signal to thevoice regeneration circuit 19 so that the robot 1 outputs a voicesaying, “You had an ordinary day. Tomorrow will be a good day,” (Step420), and the CPU returns.

When C<0 at the step 418, the CPU 16 judges that the user has not spentthe day actively. In this case, the CPU 16 outputs a control signal tothe voice regeneration circuit 19 and the drive circuits 17 so that therobot 1 outputs a voice saying, “Tomorrow will be a good day,” and nods(Step 421), and returns.

FIG. 27 is a flow chart showing the sequence of the second controlprocessing by the CPU 16. First, the CPU 16 causes theradio-communication unit 110 to send a signal requesting to sendinformation to the steps counter 8 (Step 431). When receiving thesignal, the steps counter 8 sends information representing the number ofsteps in one day to the robot 1. After the radio-communication unit 110receives the information, the CPU 16 receives the information (Step432).

The CPU 16 judges whether the number of steps the user has taken in oneday are many (Step 433). More specifically, the range of the number ofsteps the user normally takes in one day is defined in advance. When thenumber of the steps contained in the received information is within therange, the CPU 16 judges that the number of the steps is a normalnumber. When the number of the steps is over the range, the CPU 16judges that the number of the steps is a large number. When the numberof the steps is below the range, the CPU 16 judges that the number ofthe steps is a small number.

When judging the number of the steps is a large number at the step 433,the CPU 16 outputs a control signal to the voice regeneration circuit 19and the drive circuits 17 so that the robot 1 outputs a voice saying,“You have walked a lot,” and behaves as if it were dancing (Step 434),and returns.

When judging the number of the steps is a normal number at the step 433,the CPU 16 outputs a control signal to the voice regeneration circuit 19so that the robot 1 outputs a voice saying, “You have walked normally.Try to walk more tomorrow,” (Step 435), and returns.

When judging the number of the steps is a small number at the step 433,the CPU 16 outputs a control signal to the voice regeneration circuit 19and the drive circuits 17 so that the robot 1 outputs a voice saying,“You are not exercising enough. Try to walk more tomorrow,” and nods(Step 436), and returns.

FIG. 28 is a flow chart showing the sequence of the third controlprocessing by the CPU 16. It is when the radio-communication unit 110receives data which are indicative of an abnormal heart rate from thewearable sensor 9 that the CPU 16 executes the third control processing.Hence, in this case, it is possible to assume that the user is in anexcited state. A cause of the user's mental excitation can be break outof fire, for instance. Noting this, as described below, the thirdcontrol processing causes the robot 1 to detect break out of fire andaccordingly act. First, based on the quantity of infrared light detectedby the infrared rays sensor 12 d, the CPU judges whether there is fire(Step 441). More specifically, the CPU 16 compares the detected quantityof infrared light with a predetermined threshold. When the detectedquantity of infrared light exceeds the predetermined threshold, the CPU16 decides that a fire has occurred. When the detected quantity ofinfrared light does not exceed the predetermined threshold, the CPU 16decides that there is no fire. Judging that there is no fire, the CPU 16outputs a control signal to the voice regeneration circuit 19 so thatthe robot 1 outputs a voice saying, “Please calm down,” (Step 442), andreturns.

Upon judgment that there is fire at the step 441, using the currentlocation detected by the CPU 16 and the map data stored in the RAM 16 b,the CPU 16 calculates the shortest route (refuge course) from thecurrent location to an emergency exit of the building (Step 443).Following this, the CPU 16 outputs a control signal to the voiceregeneration circuit 19 and the drive circuits 17 so that the robot 1repeatedly outputs a voice saying, “I will guide you to the emergencyexit,” and move along the refuge course (Step 444), and returns.

Although the portable telephone 2 collects information regarding thevolume level of the voice at which the user talks during the call usingthe portable telephone 2, the history of received calls, the history ofmade calls on the portable telephone 2 (everyday life information) whilethe steps counter 8 collects information representing the number ofsteps the user has taken during that day (everyday life information) andthe robot 1 is controlled based on these information in the thirdembodiment, this is not limiting. For example, the portable telephone 2may collect everyday life information about the user, such as thehistory of received and made e-mails using the portable telephone 2,which is different from the voice level, the history of received calls,the history of made calls and the number of steps taken, and the robot 1may be controlled based on this everyday life information.

Although the foregoing has described that the wearable sensor 9 collectsinformation regarding the user's heart rate (physiological information)and the robot 1 is controlled based on the physiological information inthe third embodiment, this is not limiting. For instance, the wearablesensor 9 may collect physiological information on the user, such as theuser's blood pressure, which is different from the user's heart rate,and the robot 1 may be controlled based on this physiologicalinformation.

Although the foregoing has described the third embodiment as thestructure that break out of fire is detected through the third controlprocessing and the user is guided to the emergency exit when it isjudged that there is fire, this is not limiting. For example, it may bedecided that the user's condition has abruptly changed based on that theuser's heart rate is abnormal, and this may be reported to people closeto the user.

FOURTH EMBODIMENT

FIG. 29 is a block diagram showing an essential structure of a fourthembodiment of the robot according to the present invention. In FIG. 29,denoted at 10 is an air conditioner and denoted at 11 is a televisionset. Radio-communication units (not shown) having a communicationfunction such as PHS data communication or Bluetooth are incorporated inthe air conditioner 10 and the television set 11. Hence, by means ofthese radio-communication units, the air conditioner 10 and thetelevision set 11 can communicate with the radio-communication unit 110of the robot 1.

When manipulated by the user, the air conditioner 10 and the televisionset 11 transmit from their radio-communication units data whichcorrespond to the content of the manipulation such as turning on of theair conditioner and designation of a television channel. Further, uponreceipt of data regarding weather forecast provided in data broadcastingthrough satellite broadcasting for example, the television set 11 cansend the data from its radio-communication unit in response tomanipulation given by the user.

The air conditioner 10 and the television set 11 have displays whichturn on and notify the user of clogging of a filter, a failure or thelike upon occurrence of these. When there is clogging of a filter, afailure or the like, the displays turn on and the radio-communicationunits transmit a signal which notifies of the trouble.

Operation data for manipulating the air conditioner 10 and thetelevision set 11 are stored in the ROM 16 a of the robot 1. As therobot 1 sends the operation data from the radio-communication unit 110,the air conditioner 10 and the television set 11 can be remotelyoperated.

The other structures of the robot 1 according to the fourth embodimentare similar to the structure of the robot 1 according to the thirdembodiment, and therefore, will be denoted at the same reference symbolsbut will not be described.

Operations of the robot 1 upon occurrence of clogging of a filter, afailure, etc., with the air conditioner 10 and the television set 11will now be described. When the filter of the air conditioner 10 getsclogged, or when the air conditioner 10 or the television set 11 breaksdown, a signal notifying the state of the apparatus is transmitted fromthe air conditioner 10 or the television set 11. The radio-communicationunit 110 receives the transmitted signal which will be then supplied tothe CPU 16. The CPU 16 outputs a control signal corresponding to thesignal to the voice regeneration circuit 19 so that a voicecorresponding to the signal comes out from the speaker 12 c. After avoice signal corresponding to the control signal is outputted from thevoice regeneration circuit 19, a voice is emitted from the speaker 12 c.

When the user is far away, the robot 1 moves close to the user andnotifies the user of clogging of filter of the air conditioner 10 or thefailure of the air conditioner 10 or the television set 11.

Operations of the robot 1 for remotely manipulating the air conditioner10 or the television set 11 will now be described. Manipulated by theuser, the air conditioner 10 or the television set 11 sends dataexpressing the content of the manipulation given, from theradio-communication unit. The transmitted data are received by theradio-communication unit 110 of the robot 1 and then fed to the CPU 16.

The CPU 16 stores such data in the RAM 16 b and executes the followingprocessing, to thereby remotely operate the air conditioner 10 and thetelevision set 11. Based on the data, the CPU 16 judges whether the samemanipulation was given within 30 min. around the predetermined timeeveryday over five days, e.g., whether the air conditioner 10 wasstarted to operate between 06:45 and 07:15 everyday over five days.

For instance, if the air conditioner 10 was started to operate between06:45 and 07:15 everyday over five days, the CPU 16 decides that the airconditioner 10 was started to operate at 7 o'clock everyday over thefive days and waits until 7 o'clock. At 7 o'clock, the CPU 16 sends tothe air conditioner 10 from the radio-communication unit 110 operationdata which instruct to start cooling the air, whereby the airconditioner 10 is remotely manipulated.

FIG. 30 is a flow chart showing one example of the sequence ofprocessing in the CPU 16 for having the robot remotely manipulate thetelevision set 11. When deciding through the processing described abovethat data from the same data broadcasting was received at the same timeof a day everyday over five days, the CPU 16 judges whether that timehas come (Step 501). When it is not that time yet, the CPU 16 repeatsthe step 501. When it is that time, the CPU 16 sends to the televisionset 11 from the radio-communication unit 110 operation data whichinstruct to receive data from the data broadcasting (Step 502), andfurther sends to the television set 11 from the radio-communication unit110 operation data which instruct to send the received data (Step 503).

The television set 11 at the radio-communication unit thereof receivesthese operation data and operates in accordance with the operation data.From the radio-communication unit of the television set 11, the datafrom the data broadcasting are transmitted. This data are received bythe radio-communication unit 110 and fed to the CPU 16. The CPU 16accepts the data (Step 504), and outputs a control signal in accordancewith the data to the voice regeneration circuit 19 so that a voicetelling the content expressed by the data comes out from the speaker 12c (Step 505). The procedure then ends.

In this structure, if the television set 11 receives data regardingweather forecast at 7 o'clock everyday for example, the robot 1 outputsa voice notifying the content of the weather forecast at 7 o'clock.

Although the foregoing has described the fourth embodiment as thestructure that the robot 1 notifies the user of occurrence of cloggingof a filter, a failure or the like in the air conditioner 10 and thetelevision set 11 and remotely operates the air conditioner 10 and thetelevision set 11, this is not limiting. For instance, the robot 1 maynotify the user of occurrence of malfunction in a different apparatussuch as a refrigerator which is different from the air conditioner 10and the television set 11. Further alternatively, the robot 1 mayremotely operate a different apparatus such as an air purificationsystem which is different from the air conditioner 10 and the televisionset 11.

While the foregoing has described that if the television set 11 receiveddata from a particular sender approximately at the same time of a dayover more than one day, the television set 11 receives data from thesender at the time corresponding to the time of a day mentioned aboveafter these past couple of days and the robot 1 outputs a voicecorresponding to the data in the fourth embodiment, this is notlimiting. If a different apparatus such as a radio set which isdifferent from the television set 11 receives data from a particularsender approximately at the same time of a day over more than one day,the apparatus may receive data from the sender and the robot 1 mayoutput a voice corresponding to the data.

As this invention may be embodied in several forms without departingfrom the spirit of essential characteristics thereof, the presentembodiments are therefore illustrative and not restrictive, since thescope of the invention is defined by the appended claims rather than bythe description preceding them, and all changes that fall within metesand bounds of the claims, or equivalence of such metes and boundsthereof are therefore intended to be embraced by the claims.

1. A robot system comprising: a communication apparatus of one of aportable telephone and a radio communication unit for communicating withan outside communication device to receive a voice signal and output oneof a voice and a voice signal which corresponds to the received voicesignal; and a robot; the robot having: a movable unit; an imaging unit;and a control unit which drives said movable unit based on an evaluationvalue obtained by calculating the one of the voice and the voice signaloutputted from said communication apparatus and an image captured bysaid imaging unit.
 2. The robot system according to claim 1 wherein aportion of the robot has features that correspond to those of a humanand said control unit drives said movable unit to operate the robot toperform at least one of the acts of blinking, opening/closing the mouthand nodding.
 3. A robot system comprising: a communication apparatus ofone of a portable telephone and a radio communication unit forcommunicating with an outside communication device to receive a voicesignal and output one of a voice and a voice signal which corresponds tothe received voice signal; and a robot; the robot having: a movableunit; a voice input unit; a voice output unit; a control unit whichdrives said movable unit based on an evaluation value obtained bycalculating the one of the voice and the voice signal outputted fromsaid communication apparatus; and a voice determining unit whichdetermines a voice to be outputted based on the voice which is inputtedat said voice input unit, wherein said voice output unit outputs thevoice which is determined by said voice determining unit.
 4. The robotsystem according to claim 7, wherein said robot further comprises animaging unit, and said control unit drives said movable unit based on animage captured by said imaging unit.
 5. The robot system which operatesa robot based on emotions thereof according to claim 2, furthercomprising: an emotion determining unit which determines the emotion ofthe robot based on external stimulation; a state determining unit whichdetermines the current interaction state of said robot based on externalstimulation; and a reply operation control unit which causes said robotto realize an emotion for the present which is determined by saidemotion determining unit and an interactional action for the presentwhich is determined by said state determining unit.
 6. The robot systemaccording to claim 3 wherein a portion of the robot has features thatcorrespond to those of a human and said control unit drives said movableunit to operate the robot to perform at least one of the acts ofblinking, opening/closing the mouth and nodding.
 7. A robot systemcomprising a communication apparatus and a robot, the communicationapparatus having: a first receiving unit which receives from outsidefirst action instructing information which expresses an instructionregarding an action of said movable unit; a first transmission unitwhich transmits to said robot second action instructing informationwhich corresponds to said first action instructing information which isreceived by said first receiving unit; a second receiving unit whichreceives first image information which expresses an image which istransmitted from said robot; and a second transmission unit whichtransmits to outside second image information which corresponds to saidfirst image information which is received by said second receiving unit,the robot having: a movable unit; an imaging unit; a third receivingunit which receives said second action instructing information which istransmitted from said communication apparatus; a control unit whichdrives said movable unit based on said second action instructinginformation which is received by said third receiving unit; and a thirdtransmission unit which transmits said first image information to saidcommunication apparatus based on an image captured by said imaging unit.8. A robot system comprising: a communication apparatus which receives avoice signal and outputs one of a voice and a voice signal whichcorresponds to the received voice signal; and a robot; the robot having:a movable unit; an imaging unit; and a control unit which drives saidmovable unit based on the one of the voice and the voice signaloutputted from said communication apparatus and an image captured bysaid imaging unit; wherein said communication apparatus furthercomprises: a first receiving unit which receives from outside firstaction instructing information which expresses an instruction regardingan action of said movable unit; a first transmission unit whichtransmits to said robot second action instructing information whichcorresponds to said first action instructing information which isreceived by said first receiving unit; a second receiving unit whichreceives first image information which expresses an image which istransmitted from said robot; and a second transmission unit whichtransmits to outside second image information which corresponds to saidfirst image information which is received by said second receiving unit,the robot further comprises: a third receiving unit which receives saidsecond action instructing information which is transmitted from saidcommunication apparatus; and a third transmission unit which transmitssaid first image information to said communication apparatus based onthe image captured by said imaging unit, and said control unit drivessaid movable unit based on said second action instructing informationwhich is received by said third receiving unit.
 9. A robot systemcomprising: a communication apparatus which receives a voice signal andoutputs one of a voice and a voice signal which corresponds to thereceived voice signal; and a robot; the robot having: a movable unit; avoice input unit; a voice output unit; a control unit which drives saidmovable unit based on the one of the voice and the voice signaloutputted from said communication apparatus; and a voice determiningunit which determines a voice to be outputted based on the voice whichis inputted at said voice input unit, wherein said voice output unitoutputs the voice which is determined by said voice determining unit;wherein said robot further comprises an imaging unit, and said controlunit drives said movable unit based on an image captured by said imagingunit; wherein said communication apparatus further comprises: a firstreceiving unit which receives from outside first action instructinginformation which expresses an instruction regarding an action of saidmovable unit; a first transmission unit which transmits to said robotsecond action instructing information which corresponds to said firstaction instructing information which is received by said first receivingunit; a second receiving unit which receives first image informationwhich expresses an image which is transmitted from said robot; and asecond transmission unit which transmits to outside second imageinformation which corresponds to said first image information which isreceived by said second receiving unit, the robot further comprises: athird receiving unit which receives said second action instructinginformation which is transmitted from said communication apparatus; anda third transmission unit which transmits said first image informationto said communication apparatus based on the image captured by saidimaging unit, and said control unit drives said movable unit based onsaid second action instructing information which is received by saidthird receiving unit.
 10. A robot system comprising: a communicationapparatus which receives a voice signal and outputs one of a voice and avoice signal which corresponds to the received voice signal; and arobot; the robot having: a movable unit; a voice input unit; a voiceoutput unit; a control unit which drives said movable unit based on theone of the voice and the voice signal outputted from said communicationapparatus; and a voice determining unit which determines a voice to beoutputted based on the voice which is inputted at said voice input unit,wherein said voice output unit outputs the voice which is determined bysaid voice determining unit; wherein said robot further comprises animaging unit, and said control unit drives said movable unit based on animage captured by said imaging unit; further comprising: a voiceexistence determining unit which judges whether there is an input voicein predetermined cycles based on a voice signal which is inputted atsaid voice input unit; a speaker existence determining unit which judgeswhether there is a speaker in predetermined cycles based on an imagesignal which is inputted at said imaging unit; and a determining unitwhich determines the current interaction state of said robot based onthe judgment result obtained at said voice existence determining unitand the judgment result obtained at said speaker existence determiningunit.
 11. The robot system according to claim 10, further comprising: adarkness detector which judges whether it is dark around inpredetermined cycles based on the image signal which is inputted at saidimaging unit; and a determining unit which determines the currentinteraction state of said robot based on the judgment result obtained atsaid voice existence determining unit, the judgment result obtained atsaid speaker existence determining unit, and the judgment resultobtained at said darkness detector.
 12. A robot system comprising: acommunication apparatus which receives a voice signal and outputs one ofa voice and a voice signal which corresponds to the received voicesignal; and a robot; the robot having: a movable unit; a voice inputunit; a voice output unit; a control unit which drives said movable unitbased on the one of the voice and the voice signal outputted from saidcommunication apparatus; and a voice determining unit which determines avoice to be outputted based on the voice which is inputted at said voiceinput unit, wherein said voice output unit outputs the voice which isdetermined by said voice determining unit; wherein said robot furthercomprises an imaging unit, and said control unit drives said movableunit based on an image captured by said imaging unit; furthercomprising: a first detector which detects a factor which changes anemotion, based on a voice signal which is inputted at said voice inputunit; a second detector which detects a factor which changes an emotion,based on an image signal which is inputted at said imaging unit; and adetermining unit which determines the emotion of the robot based on thejudgment results obtained at said first and said second detectors. 13.The robot system according to claim 12, wherein said first detectorcomprises a unit which judges the voice level of an input voice inpredetermined cycles based on the voice signal which is inputted at saidvoice input unit, and said second detector comprises a unit which judgeswhether a face is detected in predetermined cycles based on the imagesignal which is inputted at said imaging unit.
 14. A robot systemcomprising: a communication apparatus which receives a voice signal andoutputs one of a voice and a voice signal which corresponds to thereceived voice signal; and a robot; the robot having: a movable unit; avoice input unit; a voice output unit; a control unit which drives saidmovable unit based on the one of the voice and the voice signaloutputted from said communication apparatus; and a voice determiningunit which determines a voice to be outputted based on the voice whichis inputted at said voice input unit, wherein said voice output unitoutputs the voice which is determined by said voice determining unit;wherein said robot further comprises an imaging unit, and said controlunit drives said movable unit based on an image captured by said imagingunit; further comprising: an emotion determining unit which determinesthe emotion of the robot based on external stimulation; a statedetermining unit which determines the current interaction state of saidrobot based on external stimulation; and a reply operation control unitwhich causes said robot to realize an emotion for the present which isdetermined by said emotion determining unit and an interactional actionfor the present which is determined by said state determining unit;wherein said emotion determining unit comprises: a first detector whichdetects a factor which changes an emotion, based on a voice signal whichis inputted at said voice input unit; a second detector which detects afactor which changes an emotion, based on an image signal which isinputted at said imaging unit; and a determining unit which determinesthe emotion of the robot based on the judgment results obtained at saidfirst and said second detectors.
 15. A robot system comprising: acommunication apparatus which receives a voice signal and outputs one ofa voice and a voice signal which corresponds to the received voicesignal; and a robot; the robot having: a movable unit; a voice inputunit; a voice output unit; a control unit which drives said movable unitbased on the one of the voice and the voice signal outputted from saidcommunication apparatus; and a voice determining unit which determines avoice to be outputted based on the voice which is inputted at said voiceinput unit, wherein said voice output unit outputs the voice which isdetermined by said voice determining unit; wherein said robot furthercomprises an imaging unit, and said control unit drives said movableunit based on an image captured by said imaging unit; furthercomprising: an emotion determining unit which determines the emotion ofthe robot based on external stimulation; a state determining unit whichdetermines the current interaction state of said robot based on externalstimulation; and a reply operation control unit which causes said robotto realize an emotion for the present which is determined by saidemotion determining unit and an interactional action for the presentwhich is determined by said state determining unit; further comprising:a voice existence determining unit which judges whether there is aninput voice in predetermined cycles based on a voice signal which isinputted at said voice input unit; a speaker existence determining unitwhich judges whether there is a speaker in predetermined cycles based onan image signal which is inputted at said imaging unit; and adetermining unit which determines the current interaction state of saidrobot based on the judgment result obtained at said voice existencedetermining unit and the judgment result obtained at said speakerexistence determining unit.
 16. A robot system comprising: acommunication apparatus which receives a voice signal and outputs one ofa voice and a voice signal which corresponds to the received voicesignal; and a robot; the robot having: a movable unit; a voice inputunit; a voice output unit; a control unit which drives said movable unitbased on the one of the voice and the voice signal outputted from saidcommunication apparatus; and a voice determining unit which determines avoice to be outputted based on the voice which is inputted at said voiceinput unit, wherein said voice output unit outputs the voice which isdetermined by said voice determining unit; wherein said robot furthercomprises an imaging unit, and said control unit drives said movableunit based on an image captured by said imaging unit; furthercomprising: a first detector which detects a factor used for selectingthe reply operation table, based on a voice signal which is inputted atsaid voice input unit; a second detector which detects a factor used forselecting the reply operation table, based on an image signal which isinputted at said imaging unit; and a determining unit which determinesthe reply operation table based on the judgment results obtained at saidfirst and said second detectors.
 17. The robot system according to claim16, wherein said first detector comprises a unit which judges the voicelevel of an input voice in predetermined cycles based on the voicesignal which is inputted at said voice input unit, and said seconddetector comprises a unit which judges whether a face is detected inpredetermined cycles based on the image signal which is inputted at saidimaging unit.
 18. A robot system comprising: a communication apparatuswhich receives a voice signal and outputs one of a voice and a voicesignal which corresponds to the received voice signal; and a robot; therobot having: a movable unit; a voice input unit; a voice output unit; acontrol unit which drives said movable unit based on the one of thevoice and the voice signal outputted from said communication apparatus;and a voice determining unit which determines a voice to be outputtedbased on the voice which is inputted at said voice input unit, whereinsaid voice output unit outputs the voice which is determined by saidvoice determining unit; further comprising: a reply operation tabledetermining unit which determines the reply operation table of the robotbased on external stimulation; a state determining unit which determinesthe current interaction state of said robot based on externalstimulation; and a reply operation control unit which causes said robotbased on the reply operation table for the present which is determinedby said reply operation table determining unit and an interactionalaction for the present which is determined by said state determiningunit.
 19. The robot system according to claim 18, wherein said replyoperation table determining unit comprises: a first detector whichdetects a factor which changes a reply operation table, based on a voicesignal which is inputted at said voice input unit; a second detectorwhich detects a factor which changes a reply operation table, based onan image signal which is inputted at said imaging unit; and adetermining unit which determines the reply operation table of the robotbased on the judgment results obtained at said first and said seconddetectors.
 20. The robot system according to claim 18, furthercomprising: a voice existence determining unit which judges whetherthere is an input voice in predetermined cycles based on a voice signalwhich is inputted at said voice input unit; a speaker existencedetermining unit which judges whether there is a speaker inpredetermined cycles based on an image signal which is inputted at saidimaging unit; and a determining unit which determines the currentinteraction state of said robot based on the judgment result obtained atsaid voice existence determining unit and the judgment result obtainedat said speaker existence determining unit.
 21. A robot comprising: amovable unit; a motor which is disposed to said movable unit; a drivecircuit which drives said motor; an imaging unit; an input unit whichaccepts one of a voice and a voice signal from a communication apparatusof one of a portable telephone and a radio communication unit forcommunicating with an outside communication device to receive a voicesignal and output one of a voice and a voice signal which corresponds tothe received voice signal; and a control unit which drives said movableunit based on an evaluation value obtained by calculating the one of thevoice and the voice signal accepted at said input unit and an imagecaptured by said imaging unit.
 22. The robot according to claim 21,further comprising: a receiving unit which receives action instructinginformation which expresses an instruction regarding an action of saidmovable unit; and a transmission unit which transmits said imageinformation containing an image captured by said imaging unit, whereinsaid control unit drives said movable unit based on said actioninstructing information which is received by said receiving unit. 23.The robot according to claim 21 whereby an operation program is acquiredto a memory from an external server apparatus through a communicationnetwork.
 24. The robot according to claim 21, further comprises aprogrammable controller which can modify by means of program a hardwarestructure which is for supplying a control signal to said drive circuit.25. The robot as claimed in claim 21, wherein a portion of the robot hasfeatures that correspond to those of a human and said control unitdrives said movable unit to operate the robot to perform at least one ofthe acts of blinking, opening/closing the mouth and nodding.
 26. A robotcomprising: a movable unit; a motor which is disposed to said movableunit; a drive circuit which drives said motor; a voice input unit; avoice output unit; an input unit which accepts one of a voice and avoice signal from a communication apparatus of one of a portabletelephone and a radio communication unit for communicating with anoutside communication device to receive a voice signal and output one ofa voice and a voice signal which corresponds to the received voicesignal; a control unit which drives said movable unit based on anevaluation value obtained by calculating the one of the voice and thevoice signal accepted at said input unit; and a voice determining unitwhich determines a voice to be outputted based on the voice which isinputted at said voice input unit, wherein said voice output unitoutputs the voice determined by said voice determining unit.
 27. Therobot according to claim 26 further comprising an imaging unit, whereinsaid control unit drives said movable unit based on an image captured bysaid imaging unit.
 28. The robot according to claim 27, furthercomprising: a receiving unit which receives action instructinginformation which expresses an instruction regarding an action of saidmovable unit; and a transmission unit which transmits said imageinformation containing an image captured by said imaging unit, whereinsaid control unit drives said movable unit based on said actioninstructing information which is received by said receiving unit. 29.The robot according to claim 26 whereby an operation program is acquiredto a memory from an external server apparatus through a communicationnetwork.
 30. The robot according to claim 26, further comprises aprogrammable controller which can modify by means of program a hardwarestructure which is for supplying a control signal to said drive circuit.31. The robot as claimed in claim 26, wherein a portion of the robot hasfeatures that correspond to those of a human and said control unitdrives said movable unit to operate the robot to perform at least one ofthe acts of blinking, opening/closing the mouth and nodding.
 32. A robotcomprising: a movable unit; a motor which is disposed to said movableunit; a drive circuit which drives said motor; an imaging unit forcapturing an image; a control unit that controls the movable unit tolocate a user of the robot at the center of an image on the basis of acaptured image; a receiving unit which receives from an outside deviceby one of a portable telephone and a radio communication unit actioninstructing information, which expresses an instruction regarding anaction of said movable unit; said control unit also operating to drivesaid movable unit based on said action instructing information which isreceived by said receiving unit; and a transmission unit which transmitsby one of a portable telephone or radio communication unit to theoutside device said image information containing an image captured bysaid imaging unit.
 33. The robot according to claim 32 whereby anoperation program is acquired to a memory from an external serverapparatus through a communication network.
 34. The robot according toclaim 32, further comprises a programmable controller which can modifyby means of program a hardware structure which is for supplying acontrol signal to said drive circuit.
 35. A robot comprising: a movableunit; a motor which is disposed to said movable unit; a drive circuitwhich drives said motor; a voice output unit; a receiving unit whichreceives by one of a portable telephone and a radio communication unitinformation related to everyday life of a target person from an outsidedevice; and a voice determining unit which determines a voice based onthe information received by said receiving unit, wherein said voiceoutput unit outputs the voice determined by said voice determining unit.36. The robot according to claim 35 whereby an operation program isacquired to a memory from an external server apparatus through acommunication network.
 37. The robot according to claim 35, furthercomprises a programmable controller which can modify by means of programa hardware structure which is for supplying a control signal to saiddrive circuit.
 38. A robot comprising: a movable unit; a motor which isdisposed to said movable unit; a drive circuit which drives said motor;a memory unit which stores manipulation history information regardingthe history of manipulation given by a target person to an externalapparatus; and an operation unit which manipulates said externalapparatus based on said manipulation history information stored in saidmemory unit.
 39. The robot according to claim 38, wherein said operationunit performs operations of: manipulating said external apparatus, whensaid manipulation history information is information which indicatesthat information was received from a particular sender approximately atthe same time of a day over a couple of days; causing receipt ofinformation from said particular sender at the time corresponding tosaid time of a day after said couple of days; and causing transmissionof said information which is received, and the robot further comprises:a receiving unit which receives said information transmitted from saidexternal apparatus; and a voice output unit which outputs a voice basedon the information received by said receiving unit.
 40. The robotaccording to claim 38 whereby an operation program is acquired to amemory from an external server apparatus through a communicationnetwork.
 41. The robot according to claim 38, further comprises aprogrammable controller which can modify by means of program a hardwarestructure which is for supplying a control signal to said drive circuit.